
\documentclass[11pt]{article}
\newtheorem{define}{Definition}
%\newcommand{\E}{{\mathbb{E}}}
%\usepackage{psfig}
\usepackage{amsmath,amssymb,amsfonts}
\usepackage{fullpage, color}

\oddsidemargin=0.15in
\evensidemargin=0.15in
\topmargin=-.5in
\textheight=9in
\textwidth=6.25in
\multlinegap=140pt

\begin{document}
\input{preamble.tex}

%{lecture number}{lecture date}{Elena Grigorescu}{Your name}
\lecture{18}{Oct 18, 2012}{Elena Grigorescu}{Gowtham Kaki}

%%%% body goes in here %%%%

Today we'll introduce locally testable codes(LTCs), see connections with other  objects studied in complexity theory (PCPs), and introduce the Fourier analytic technique that we will use in the next lecture to show that the Hadamard code is a locally testable code. 
\section{Intro to Locally Testable Codes}

LTCs are error-correcting codes that can admit fast membership testers.
To motivate these codes, first recall that given an \((n,{ k },d)\)-error correcting code  and a received word $r$ one can uniquely decode it when it has at most \(\frac { d }{ 2 } \) errors. If more errors occur then a classical algorithm would still run the decoding procedure but have no guarantees about the output. In a previous lecture we saw a decoding algorithm that was efficient (it ran in linear time), but this would still be huge waste of time if by the end we realize that no codeword is output. 
We would be able to avoid this wasteful computation if we could quickly test whether the received word is close to some codeword. This is exactly what a locally testable code is: one for which we could test in sub-linear time if a received word is close to a codeword or not.
We remark that though not all codes are locally testable, most of the codes commonly used in communication and storage are.

\begin{definition}[Locally Testable Code]
Let \(C=\{{ C }_{ n }\}_{n\rightarrow \infty}\) where 
$C_n$ is an \((n,{ k }_{ n },d_{ n })\)  error correcting code. Then, \(C\) is 
\((k,{ \epsilon  }_{ 1 },{ \epsilon  }_{ 2 },\delta )\)-locally testable if 
there exists a property testing algorithm \(A\) such that, when given oracle 
access to \(r\in { \{ 0,1\}^{ n }  }\), makes k queries and satisfies

(Completeness) if \(r\in C\), then accepts it with probability atleast 
\(1-{\epsilon}_{1}\)

(Soundness) if \(r\) is \(\delta \)-far from \(C\), then accepts it with
probability atmost \({\epsilon}_{2}\).

\end{definition}

So an LTC is just a testable property consisting of codewords.
Given the above definition, the following questions are interesting
\begin{itemize}
\item What codes are locally testable?
\item What parameters can be achieved for locally testable codes?
\end{itemize}

As already mentioned, in the next lecture we will see an example of a LTC. 
LTCs are combinatorial objects related to Probabilistically
Checkable Proofs (PCPs). While no formal reductions between these two objects  is known,  currently the settings of parameters that can be achieved for these objects (number of queries vs length) are very similar. 
We will next give just a quick flavor of what PCPs are, to give some intuition about the connection with LTCs. 


\begin{definition}[Deterministic Polynomial Time Verifier]
A deterministic polynomial time verifier \(V\) for a language \(L\) is a an algorithm running in polynomial time 
 that takes as input a string \(x\) and a proof \(\pi\) and accepts/rejects.
 
\end{definition}
 The language verified (not decided) by $V$ is \(L=\{x | \exists \pi, V(x,\pi )=ACCEPT\}\).
 So, $V$  satisfies the following properties

(Completeness) \(\forall x\in L\), \(\exists \pi \) such that \( V(x,\pi )\) returns ACCEPT

(Soundness) \(\forall x\notin L\), \(\forall \pi, V(x,\pi )\) returns REJECT

So $NP=$ class of all languages $L(V)$ for some deterministic verifier $V$.



\begin{definition}[Randomized Verifier]
An \((r,k)\)- verifier \(V\) for a language \(L\) is a randomized 
algorithm that
\begin{enumerate}
\item  Takes as input a string \(x\)  and a proof \(\pi\) to which it has oracle access, 
\item  Makes $r$ random choices,
\item Makes $k$ queries to \(\pi\), and 
\item Accepts/rejects.
\end{enumerate}
\end{definition}
The language defined by $V$ is $L$ s.t. 

(Completeness) \(\forall x\in L\), \(\exists \pi \) such that \( V^{\pi}(x) \) always returns ACCEPT;

(Soundness) \(\forall x\notin L\), \(\forall \pi, V^{\pi}(x)\) returns REJECT with probability $> \frac { 1 }{ 2 }$


The proof $\pi$ is a probabilistically checkable proof.
$PCP[r,k]=$ class of languages that have  an $(r,k)$-verifier.

In particular, the class of languages decidable with no coin flips and no queries to a proof form $P$. 
 Also the languages decidable with no randomness and a $poly(n)$ queries are exactly those in NP. (I.e. $PCP(0, poly(n))=NP$.)
The PCP theorem states one of the most surprising results in computational complexity: any language in NP has a verifier that makes only a constant number of queries to a proof and tosses very few coins.

\begin{theorem}[PCP Theorem] \cite{AroraLMSS98,Dinur07}
$NP = PCP(O(\log n),O(1)))$
\end{theorem}


We will next proceed with introducing the Fourier analytic method, which is a technique with numerous applications in complexity theory, including applications to the proof of the PCP theorem.

\section{Fourier Analysis Of Boolean Functions}

The Fourier analytic method has been very useful in the study of Boolean functions. The idea is to express a function in a particularly nice basis, the Fourier basis.

We will study Boolean functions  defined as
\begin{center}
 $f : { Z }_{ 2 }^{ n }  \to { Z }_{ 2 }$.
\end{center}
It is sometimes more convenient to make the notation switch to a real valued function with $\pm 1$ values: i.e replace bit $b$ by $(-1)^b$ (so $0\mapsto 1$ and $1\mapsto 0$) and now the function is
\begin{center}
 $f : { Z }_{ 2 }^{ n }  \to \{-1,1\}$
\end{center}

\paragraph{The Fourier basis} For all $a\in { Z }_{ 2 }^{ n }$ Let us define a boolean function ${\chi}_{a} $ as follows
\begin{center}
 ${\chi}_{a}(x)={ (-1) }^{ a\cdot x }$ 
\end{center}
where $a\cdot x$ denotes inner product of vectors $a$ and $x$ $\mod 2$. We will later show that the vectors $\{\chi_a\}_{a\in \Z_2^n}$ form a basis of $R^{2^n}$ (the real vector space of dimension $2^n$.)


We will proceed to show useful properties of ${\chi}_{a}$.

\begin{proposition}
$\E_{x\in Z_2^n} \chi_a(x)=\begin{cases}
1, \mbox{ if } a=0\\
0, \mbox{ ow }
\end{cases}$
\end{proposition}
\begin{proof}  When $a=\bar{0}$, then RHS reduces to \(\frac { 1 }{ { 2 }^{ n } } \sum _{}{1}\) = 1.

If $a\not =0$, let $i$ be s.t. $a_i\not=0$, then we can pair up any $x$ with $x+ e_i$ since $a\cdot x \not = a\cdot (x+e_i)$, and so ${ { (-1) }^{ a\cdot{x} }+{ (-1) }^{ a\cdot{(x +e_i )}} = 0 }$. 
\end{proof}

\begin{proposition}
 $\forall a,x,y\in{ z }_{ 2 }^{ n }$, ${\chi}_{a}(x).{\chi}_{a}(y) = {\chi}_{a}(x+y)$
\end{proposition}
\begin{proof}
 Trivially follows from the the additive property of exponents.
\end{proof}

\begin{proposition}
 $\forall a,b,x\in{ z }_{ 2 }^{ n }$, ${\chi}_{a}(x).{\chi}_{b}(x) = {\chi}_{a+b}(x)$
\end{proposition}
\begin{proof}
 Trivially follows from the the additive property of exponents.
\end{proof}
% \begin{proposition}[Expectation of ${\chi}_{a}$]
% The expectation of ${\chi}_{a}$ over the boolean vector space ${ Z }_{ 2 }^{ n }$ 
% \begin{center}
%  \(\E_{x \in { Z }_{ 2 }^{ n }}[{\chi}_{a}(x)] = 
%  \left\{
%     \begin{array}{ll}
%         1  & \mbox{if } a = \bar{ 0 } \\
%         0  & \mbox{otherwise} 
%     \end{array}
% \right.
%  \)
% \end{center}
% \end{proposition}
% \begin{proof}
% It is easy to see that\\
% \begin{center}
%  \(\E_{x \in { Z }_{ 2 }^{ n }}[{\chi}_{a}(x)]\) = \(\frac { 1 }{ { 2 }^{ n } } \sum _{ x\in { Z }_{ 2 }^{ n } }{ { (-1) }^{ a.x } } \)\\
% \end{center}
%  When $a\neq \bar{0}$, then for every ${x}_{i}\in{ Z }_{ 2 }^{ n }$, there exists a unique ${x}_{j}$ such that \\
%  \begin{center}
%   ${x}_{j} = {x}_{i}$ $XOR$ ${2}^{n-1}$.\\
%  \end{center}
%  Now, it is easy to see that $|a.{x}_{i} - a.{x}_{j}| = 1$. It follows that ${ { (-1) }^{ a.{x}_{i} }+{ (-1) }^{ a.{x}_{i} } = 0 }$. There are ${ 2 }^{ n-1 }$
%  such descrete pairs of vectors in ${ Z }_{ 2 }^{ n }$ each of which produce a sum of 0. Hence, \(\E_{x \in { Z }_{ 2 }^{ n }}[{\chi}_{a}(x)] = 0\)
% \end{proof}

\begin{definition}[Inner Product]
 The inner product
 on pairs of functions $f, g:\Z_2^n\rightarrow \pm 1$ is
 \begin{center}
  $\left< f,g \right>$ = \(\frac { 1 }{ { 2 }^{ n } } \sum _{ x\in { Z }_{ 2 }^{ n } }{ f(x)\cdot g(x) }=\E_{x \in { Z }_{ 2 }^{ n }}(f(x)\cdot g(x)) \)\\
 \end{center} 
  It follows that the ${\ell}_{2}-norm$ of f is
 \begin{center}
 $\left\|f\right\|$ = $\sqrt { \E_{}[{f}^{2}]}$ = $\sqrt {\left< f,f \right>}$ 
 \end{center}
\end{definition}

\begin{proposition}
  $ \{ { \chi }_{ a }\}_{ a \in { Z }_{ 2 }^{ n }}$ forms an orthonormal basis of ${\mathbb{R}}^{{2}^{n}}$
\end{proposition}
\begin{proof}
\begin{center}
 ${\left\|{ \chi }_{ a }\right\|}^{2}$ = $\left< { \chi }_{ a },{ \chi }_{ a } \right>$ = 
  \(\frac { 1 }{ { 2 }^{ n } } \sum _{ x\in { Z }_{ 2 }^{ n } }{{{ \chi }_{ a }(x)}^{2}} \) = 1.
\end{center}

\begin{center}
 $\left< { \chi }_{ a },{ \chi }_{ b } \right>$ = 
  \(\frac { 1 }{ { 2 }^{ n } } \sum _{ x\in { Z }_{ 2 }^{ n } }{{ \chi }_{ a+b }(x)} \) = 0, whenever $a \neq b$ (follows from proposition 5).
\end{center}
So, they are orthogonal.
Since these are $2^n$ orthonormal vectors in $R^{2^n}$ they must form a basis.
\end{proof}


\begin{definition}[Fourier Expansion of f]
Every function \(f:\{ -1,1\} ^{ n }\rightarrow \mathbb{R}\) can be uniquely expanded as
\begin{center}
  \(f\quad =\quad \sum _{ a\in { Z }_{ 2 }^{ n } }^{  }{ { \hat { f }  }_{ a } } { \chi  }_{ a }\)
\end{center}
This expansion is called Fourier expansion of $f$ and $\hat {f}_{a}$ is called Fourier coeffecient of f.
\end{definition}


\begin{proposition}
For a boolean function f and its fourier coeffecient $\hat {f}_{a}$,
\begin{center}
  \(\hat { f }_{ a } \quad = \quad \left< f,{ \chi  }_{ a } \right> \)
\end{center}
\end{proposition}
\begin{proof}
% \begin{multline*}
%   \text{\(\left< f,{ \chi  }_{ a } \right>\)\\
%   \begin{aligned}
%     &= \text{\(\E_{x \in {Z}_{2}^{n}} f(x) {\chi}_{a}(x)\)} \\
%     &= \text{\(\E_{x \in {Z}_{2}^{n}} \sum _{ b\in { Z }_{ 2 }^{ n } }^{  }{ { \hat { f }  }_{ b } } { \chi  }_{ b } {\chi}_{a}(x)\)} \\
%     &= \text{\(\E_{x \in {Z}_{2}^{n}} \sum _{ b\in { Z }_{ 2 }^{ n } }^{  }{ { \hat { f }  }_{ b } } { \chi  }_{ a+b }\)} 
%   \end{aligned}
% \end{multline*}
% 
% 

\begin{eqnarray*}
\left<f, \chi_a\right> &=&\E_x f(x)\chi_a(x)=\E_x( \sum_b {\hat f}_b \chi_b(x))\chi_a(x)\\
&=& \E_x \sum_b {\hat f}_b \chi_b(x)\chi_a(x)=\E_x \sum_b {\hat f}(b) \chi_{a+b}(x)\\
&=& \sum_b {\hat f}_b\E_x \chi_{a+b}(x)={\hat f}_a,
\end{eqnarray*}

 where we used that when $a \neq b$, $\chi_{a+b}(x)=0$, and  so, the only term remaining above is when $a = b$. 
\end{proof}

\begin{theorem}[Parseval's Theorem]
For any boolean function f,
\begin{center}
  \(\left< f,f \right> = \E_{x \in { Z }_{ 2 }^{ n }} f^2= \sum _{ a\in { Z }_{ 2 }^{ n } }^{  }{ { { \hat { f }  }_{ a } }^{ 2 } } =1\)
\end{center}
\end{theorem}
\begin{proof}
\begin{center}
      \(\left< f,f \right>\) =
  \(\left<
   \sum_{ a\in Z_{ 2 }^{ n } } { \hat{ f }}_{ a }{ \chi_{ a }} ,\sum_{ a\in Z_{ 2 }^{ n } }  { \hat{ f }}_{ a } \chi_{ a }  \right> \)
                             = \(\sum_{ a,b\in { Z }_{ 2 }^{ n } }
                               \left<  {\hat f}_{ a } \chi_{ a },
                               {\hat f}_{ b} \chi_{ b } \right> \) 
                             = \(\sum_{a\in { Z }_{ 2 }^{ n }} {\hat { f }}_{ a}^{ 2 } \)
\end{center}
\end{proof}


\bibliographystyle{plain}
\bibliography{biblio}

\end{document}


